System and Method for Optimizing Testing of Software Production Incidents

ABSTRACT

This disclosure relates generally to software testing, and more particularly to a system and method for optimizing testing of software production incidents. In one embodiment, the method comprises analyzing an incident ticket using a machine learning algorithm to identify one or more keywords in the incident ticket, and identifying a location of the incident ticket based on the one or more keywords, a test workspace corresponding to the incident ticket based on the location, and a plurality of specific test cases corresponding to the incident ticket based on the test workspace. The identification leads to a first scenario and a second scenario. In the first scenario, the method further comprises initiating a learning process based on intelligence gathered from a manual processing of the incident ticket. In the second scenario, the method further comprises executing the plurality of specific test cases in a test environment.

TECHNICAL FIELD

This disclosure relates generally to software testing, and moreparticularly to a system and method for optimizing testing of softwareproduction incidents.

BACKGROUND

In the current business environment, need for quality assurance of asoftware product is paramount to the success of Information Technology(IT) organizations. Quality assurance primarily involves testing of asoftware product at various stages of the software lifecycle to minimizedefects. However, testing is a major factor driving overall cost of theproject/program and anything done to reduce the cost of testing directlyor indirectly results in cost saving to the project/program. In thiscontext, automation in the area of software testing has grown and a lotof automation techniques are now available to increase efficiency andreduce cost. For example, these techniques include test cases automationto reduce cycle time, process automation to reduce overall schedule,automation of test cases when the requirements are being developed,parallel execution along with development when coding is being done,adding services that control the updating of test cases on change,change management and their impacts on testing and tools to fix them,and so forth. All these techniques involve reducing the time taken tooverall testing, thereby reducing cost.

Additionally, the overall quality of any software product is determinedbased on the total production or post release defects that have leakedinto the product. It is preferable to identify and correct defects assoon as possible in a software production so as not to adversely impactthe customer experience or the organization's reputation andcompetitiveness. However, if the total number of production defects isvery high, the time taken and cost involved for re-rollout of a productrelease or for fixing and testing the defects in the release is achallenge as testing imposes a big bottleneck. The main reason for aboveis time and cost incurred to manually identify right test cases toverify the fix, to manually execute the identified test cases for thegiven fix, and to manually identify regression test suite to make surethat the fix has not broken any other major functionalities.

A cost analysis shows that when a defect is leaked into production thecost of fixing that defect and testing the solution is the costliest.For example, if a defect that could have been detected in requirementphase is detected post release, then it would cost about 10-100 timesmore to fix than if the defect had been detected in the requirementphase itself. Further, if a defect is leaked to release afterconstruction, then it would be 25 times costly to fix it. As statedabove, this is mostly because of the retesting effort that is needed.

Existing software testing techniques do not completely address theissues stated above particularly with respect to production or postrelease defects. Existing techniques to test the production defectsand/or to arrest production defects in customer environment involvemanual processing and operations and are therefore time consuming andcost intensive. Moreoever, there is no automated way to connect thevarious systems so as to optimize fixing and testing of softwareproduction defects. All these results in increased business spend for aparticular release.

SUMMARY

In one embodiment, a method for optimizing testing of softwareproduction incidents is disclosed. In one example, the method comprisescategorizing an incident ticket received from one or more sources basedon one or more pre-defined parameters. The incident ticket correspondsto an obstruction in a software production. The method further comprisesanalyzing the incident ticket using a machine learning algorithm toidentify one or more keywords in the incident ticket in response tocategorization. The method further comprises identifying a location ofthe incident ticket based on the one or more keywords, a test workspacecorresponding to the incident ticket based on the location, and aplurality of specific test cases corresponding to the incident ticketbased on the test workspace. The identification leads to a firstscenario and a second scenario. In the first scenario, the methodfurther comprises initiating a learning process based on intelligencegathered from a manual processing of the incident ticket. In the secondscenario, the method further comprises identifying a test environmentfor the plurality of specific test cases, and executing the plurality ofspecific test cases in the test environment.

In one embodiment, a system for optimizing testing of softwareproduction incidents is disclosed. In one example, the system comprisesat least one processor and a memory communicatively coupled to the atleast one processor. The memory stores processor-executableinstructions, which, on execution, cause the processor to categorize anincident ticket received from one or more sources based on one or morepre-defined parameters. The incident ticket corresponds to anobstruction in a software production. The processor-executableinstructions, on execution, further cause the processor to analyze theincident ticket using a machine learning algorithm to identify one ormore keywords in the incident ticket in response to categorization. Theprocessor-executable instructions, on execution, further cause theprocessor to identify a location of the incident ticket based on the oneor more keywords, a test workspace corresponding to the incident ticketbased on the location, and a plurality of specific test casescorresponding to the incident ticket based on the test workspace. Theidentification leads to a first scenario and a second scenario. In thefirst scenario, the processor-executable instructions, on execution,further cause the processor to initiate a learning process based onintelligence gathered from a manual processing of the incident ticket.In the second scenario, the processor-executable instructions, onexecution, further cause the processor to identify a test environmentfor the plurality of specific test cases, and to execute the pluralityof specific test cases in the test environment.

In one embodiment, a non-transitory computer-readable medium storingcomputer-executable instructions for optimizing testing of softwareproduction incidents is disclosed. In one example, the storedinstructions, when executed by a processor, cause the processor tocategorize an incident ticket received from one or more sources based onone or more pre-defined parameters. The incident ticket corresponds toan obstruction in a software production. The stored instructions, whenexecuted by a processor, further cause the processor to analyze theincident ticket using a machine learning algorithm to identify one ormore keywords in the incident ticket in response to categorization. Thestored instructions, when executed by a processor, further cause theprocessor to identify a location of the incident ticket based on the oneor more keywords, a test workspace corresponding to the incident ticketbased on the location, and a plurality of specific test casescorresponding to the incident ticket based on the test workspace. Theidentification leads to a first scenario and a second scenario. In thefirst scenario, the stored instructions, when executed by a processor,further cause the processor to initiate a learning process based onintelligence gathered from a manual processing of the incident ticket.In the second scenario, the stored instructions, when executed by aprocessor, further cause the processor to identify a test environmentfor the plurality of specific test cases, and to execute the pluralityof specific test cases in the test environment.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this disclosure, illustrate exemplary embodiments and, togetherwith the description, serve to explain the disclosed principles.

FIG. 1 is a block diagram of an exemplary system for optimizing testingof software production incidents in accordance with some embodiments ofthe present disclosure;

FIG. 2 is a functional block diagram of an incident management engine inaccordance with some embodiments of the present disclosure;

FIG. 3 is a flow diagram of an exemplary process for optimizing testingof software production incidents in accordance with some embodiments ofthe present disclosure;

FIG. 4 is a flow diagram of a detailed exemplary process for optimizingtesting of software production incidents in accordance with someembodiments of the present disclosure; and

FIG. 5 is a block diagram of an exemplary computer system forimplementing embodiments consistent with the present disclosure.

DETAILED DESCRIPTION

Exemplary embodiments are described with reference to the accompanyingdrawings. Wherever convenient, the same reference numbers are usedthroughout the drawings to refer to the same or like parts. Whileexamples and features of disclosed principles are described herein,modifications, adaptations, and other implementations are possiblewithout departing from the spirit and scope of the disclosedembodiments. It is intended that the following detailed description beconsidered as exemplary only, with the true scope and spirit beingindicated by the following claims.

Referring now to FIG. 1, an exemplary system 100 for optimizing testingof software production incidents is illustrated in accordance with someembodiments of the present disclosure. In particular, the system 100implements an incident management engine for providing optimized testingof incidents corresponding to defects or obstructions in a softwareproduction. As will be described in greater detail in conjunction withFIG. 2, the incident management engine interacts with multiple users(e.g., quality assurance team, development team, business analyst team,and so forth) and multiple software development, testing, and managementsystems or platforms (e.g., test management systems, test environmentsystems, and so forth), identifies a test workspace by analyzing anincident ticket, determines required test cases for the incident ticketin the test workspace, and executes the required test cases in arequisite test environment. The system 100 comprises one or moreprocessors 101, a computer-readable medium (e.g., a memory) 102, and adisplay 103. The computer-readable medium 102 stores instructions that,when executed by the one or more processors 101, cause the one or moreprocessors 101 to perform optimized testing of software productionincidents in accordance with aspects of the present disclosure. Thesystem 100 interacts with users via a user interface 104 accessible tothe users via the display 103.

Referring now to FIG. 2, a functional block diagram of an incidentmanagement engine 200 implemented by the system 100 of FIG. 1 isillustrated in accordance with some embodiments of the presentdisclosure. In some embodiments, the incident management engine 200comprises an incident logging module 201, an incident analysis module202, a test suite generation module 203, and an execution module 204. Auser may interact with the incident management engine 200 from the webbrowser or other interfaces. The incident management engine 200 mayfurther interact with one or more continuous integration systems 205,one or more test management systems 206, and one or more testenvironment systems 207.

The incident logging module 201 receives and logs all the incidentseither via a manual or an automated process whenever there is aproduction failure that has been encountered in any of the softwareproduct. Incidents may be raised by customer on phone, via Fax, or viaemail. An incident ticket is automatically generated and logged in anincident repository by the incident logging module 201 whenever anincident is reported. The incident ticket captures the information froma customer site. It should be noted that incident typically provide onlycursory information that may be captured.

The incident analysis module 202 analyzes the incident tickets so as todetermine the categories of the incident tickets and to identify a testworkspace. For example, in some embodiments, the incident tickets may becategorized based on departments they belong to. Once categorized, theincident tickets are sorted. Incident tickets that only require a L1support (e.g., telephone helpdesk, customer center support, and soforth) are moved out from here directly. The remaining incident ticketsare analyzed against the main areas/locations which are identified forthe first time with the involvement of the business analysts, qualityanalysts, and application development team. The teams provide thepriority and severity of each area. The severity and priority of thearea are the average of the all the teams. The business holds the finalveto to either accept or overwrite the whole value provided by differentteams. In some embodiments, the priority and severity rating is done inthree levels for which the threshold values may be a configurable value.By way of example, the incident logging module 201 may have defaultthreshold values for each rating which may be updated by a user. Forexample, in certain embodiments, sample illustration of the ratingthreshold values is shown in Table A below while sample illustration ofthe area/location priority values provided by different teams is shownin Table B:

TABLE A Rating Threshold Value High 1 Medium 2 Low 3

TABLE B BA QA Development Business Area/Location Team Team Team AverageTeam Pharmacy 3 1 1 1.67 1 queue Stores 2 3 3 2.67 2 Web store 3 3 3 3 3Billing 3 3 3 3 1

The test suite generation module 203 identifies the right test casesneeded for execution based on the analytical result. The right testcases are identified based on their suitability for a given incidentticket belonging to an identified test workspace. Further, the executionmodule 204 executes the identified test cases in a right testenvironment. The test environment may be tools, batch jobs, or locationswhere test cases may be executed either manually or automatically. Theincident management engine 200 may integrate with the various systems toensure that the right environment is provided to the right test cases sothat the result could be appropriate. The contiguous integration systems205 enable the incident management engine 200 to select an appropriateor a requisite environment that may be required for a specific testcase.

Additionally, the incident management engine 200 may interact with thetest management systems 206 that store and manage various test cases,defects, fixes results of execution of test cases, and so forth. Thetest management systems 206 may include one or more repositories forstoring various data. For example, it may include a test casesrepository to store various test cases, a test workspace repository tostore various test workspaces, and a location repository to storevarious locations to which incident tickets may belong to. In someembodiments, the test management systems 206 may be an applicationlifecycle management (ALM) platform such as IBM™ Rational® Jazz™platform, Microsoft™ Team Foundation Server (TFS), HP® ALM, and soforth. Alternatively, the test management systems 206 may be any testmanagement tool such as JIRA™ that may be employed for testing orproject management of the software product. Further, the incidentmanagement engine 200 may interact with the test environment systems 207that provide for the location from where the test cases are operated.The execution module 204 connects with the test environment systems 207to access the right environment to run the test cases.

In some embodiments, the incident management engine 200 provides foroptimized testing of software production incidents by performingfollowing primary functions: (i) identification of test workspaces byanalyzing the production incidents/defect data logged using keywordsearch analytics, (ii) determination of the required test cases byanalyzing the existing test cases in identified work spaces usingkeyword search analytics, and (iii) execution of the identified testcases in the requisite test environments and generating analysisreports. Each of these functions will be described in greater detailbelow.

The incident logging module 201 receives details of the productincidents or the defect data from one or more sources (e.g., such asincident reported by user on phone, via Fax, or via email). An incidentticket is created and logged in an incident repository corresponding tothe incident is reported. The incident tickets so created are thenclassified based on one or more predefined parameters such as who loggedthe incident, what system is the incident belonging to, and whichlocation did the incident come from. Thus, in some embodiments, thetickets are categorized based on the person, system and location. Forexample, in certain embodiments, sample illustration of categorizedincident tickets is provided in following table:

TABLE C Incident No. Person System Country Country_Area IncidentDescription 1 XXXXX Pharma USA Pharma_USA Pharmacy application notlaunching 2 YYYYY Pharma INDIA Pharma_INDIA Optical system does notallow payments 3 ZZZZZ Ecom UK Ecom_UK XXX home page is not opening

The categorized tickets are then analyzed based on the keywords in theincident description to separate out the L1 tickets. In someembodiments, the condition that needs to be satisfied for L1 incidentticket may be that the time taken to fix the issues is less than 10hours and the fix is purely hardware related. The identified L1incidents are separated out and given to the incident management teamfor manual fixing. The remaining categorized incident details are thenprovided to the incident analysis module 202 where the incidentdescription provided with the incident is retrieved and analyzed. Theincident description is analyzed using the artificial intelligencetechnique for identifying the different keywords in the incidentdescription text. The artificial intelligence technique may include anymachine learning algorithms such as case-based reasoning, decision treelearning, association rule learning, cluster analysis, rule-basedlearning, artificial neural networks, Bayesian networks, geneticalgorithms, fuzzy logic, inductive logic, multi-agent models,reinforcement learning, hybrid learning, and so forth. It should benoted that the keywords are specific details provided in the incidentdescription text by the customer or the operational support team whoperform the incident reporting. The keywords is then used forkeyword-location mapping, i.e., mapping the incident tickets to therespective area/location to which the ticket has to be routed based onthe priority and severity of that keyword and location. It should benoted that the priority and severity is not only based on incidentticket but also based on its impact on business. For example, in someembodiments, a sample illustration of the different locations' priorityand severity rating is provided in following table:

TABLE D Location Priority Severity Pharma Queue 1 1 Stores 2 2 Web Store3 3

The incident description is compared using the artificial intelligencetechnique with the location details available in incident analysismodule 202 such as those listed in Table D. It is compared to see if anyof the identified keywords in the incident description matches with thelocation names. If the identified keywords matches then the location ofwhere the incident ticket has to go, severity and priority are taken infrom the keyword-location mapping information available. It should benoted that the initial keyword of the mapping is based on the inputsprovided in the incident management engine 200 and the information thatis read by the machine learning algorithms. As will be appreciated, thekeyword-location mapping gets updated from time to time based on thelearning by the machine learning algorithms. For example, in someembodiments, a sample illustration of the keyword-location mapping isprovided in following table:

TABLE E Keywords Location Priority Severity Pharma, Health Pharma Queue1 1 Associate, Incentive, Plan Stores 2 2 Ecom, Homepage Web Store 3 3

If the identified keywords do not match then the incident analysismodule 202 returns the notification information to the user stating nomatch could be found. The user may then be provided with the option toenter the new location, severity, priority into the system configurationmapping details. If the user selects an existing location, then theincident analysis module 202 takes keyword as inputs and updates thekeyword-location mapping details with the corresponding keywords usingthe machine learning techniques automatically. As will be appreciated,with time and usage the module keeps learning and the keyword-locationmapping is updated with more and more keywords which may be used toassign future incident tickets.

Thus, in short, the incident analysis module 202 reads text from theincident description, searches for keywords of the text from thekeyword-location mapping, and provides location, priority, and severityif the text matches with those in mapping. However, if the text does notmatch with those in mapping, the incident analysis module 202 returnsnotification to the user, provides option to enter the new location, andmodifies the mapping details with new location and keywords.

The identified location/area details using the keywords are thenprovided to the test management system 206 to identify the testworkspaces corresponding to the location/area. The incident ticketscorresponding with the specific location are added to the testmanagement system. The mapping of the location and the work spaces aremaintained in the incident analysis module 202. It should be noted thatthe mapping is setup using manual mechanism for the first time. However,as new workspaces are created in the test management system 205, theincident management engine is fed with data of which location theworkspace is going to handle. The location-workspace mapping is updatedbased on this information. Further, it should be noted that as themapping data does not change frequently, this data is typically keptconstant. For example, in some embodiments, a sample illustration ofmapping of the location to different workspaces identified is providedin following table:

TABLE F Location Workspace Pharma Queue WS1, WS2 Stores WS4, WS3

The details of identified test workspaces are then used by the incidentanalysis module 202 and the test management system 206 to identify therequired or specific test cases for the incident tickets. In someembodiments, all the test cases identified may be further refined usinga two level keyword search analytics implemented using artificialintelligence technique. For example, for each of the location-workspacemapping the initial search keywords may be updated based on theexpertise of the quality assurance (QA) team for the initial setup. QAprovides the search keywords based on the earlier experience and buildsthe initial mapping. QA team may provide their inputs in to theremedy/incident mapping system. These keywords are the initial keywordsthat facilitate the location-workspace-keyword mapping or the searchkeyword mapping. For example, in some embodiments, a sample illustrationof the initial search keyword mapping provided by QA team is shown infollowing table:

TABLE G Location Workspace Initial Search Keywords Pharma Queue WS1, WS2Pricing, Cost Stores WS4, WS3 Costing, Financials

After the initial setup, the search keyword mapping is enhanced usingself-learning techniques implemented using artificial intelligencetechnique. The incidents that have been identified earlier may be passedthrough the defect triage process. As will be appreciated, defect triageprocess is a process in which project stakeholders go through new orexisting defects and decide on the action items. For example, during thetriage process the key secondary words may be added to the incidenttickets. As the additional keywords are provided, the machine learningalgorithm of the artificial intelligence technique learns the keywordsand keeps updating them to the existing table automatically. Thelearning process involves understanding the input supplied by the uservia machine learning algorithm, starting the artificial intelligenceanalytics to learn the keywords, and updating the keywords in the secondsearch keyword mapping based on the entry by the user. Thus, the searchkeyword mapping gets enhanced with new keywords every time it learns newinformation. For example, in some embodiments, a sample illustration ofthe enhanced search keywords mapping is provided in following table:

TABLE H Location Workspace Enhanced Search Keywords Pharma Queue WS1,WS2 Pricing, Cost, Optical Stores WS4, WS3 Costing, Financials, Workers

A test case location is then identified from the different testworkspaces identified earlier based on the location-workspace mappingand corresponding search keyword mapping. The test case location tellsthe exact location where the test cases are present. For example, insome embodiments, a sample illustration of the test case locationidentification is provided in following table:

TABLE I Test Case Location Workspace Enhanced Search Keywords LocationPharma Queue WS1, WS2 Pricing, Cost, Optical WS1 Stores WS4, WS3Costing, Financials, Workers WS3

The test management system 206 provides the identified test caselocation to the test suite generation module 203. The test suitegeneration module 203 then identifies the specific test cases from thetest case location using the keyword search technique as explainedabove. In some embodiments, the specific test cases so identified may beoptimized using optimization tools or all the specific test cases may beselected based on an option selected by the user. The identifiedspecific test cases are accordingly provided to the test suitegeneration module 203. Once the test suite generation module 203 isloaded with all the test cases resulting from the search, thetester/user may be prompted for verification. An input from the testeris taken to verify and confirm whether the identified test cases aresufficient to ensure that the test cases are good enough for thevalidation process. If the test cases are insufficient the user/testeris provided with the option to update/create/delete/modify new testsinto the system. The engine 200 would learn the information keyed in andsave them to the test management system 206 along withlocation-workspace mapping from where the test cases were picked up forautomatic handling of future incident tickets.

The specific test cases identified in the test suite generation module203 are provided to the execution module 204 for execution. Theexecution module 204 establishes connection with the test environmentsystem 207 and identifies the right environment to run the specific testcases. It should be noted that a production like environment ispreferable. Typically, the environment used may be a user acceptancetesting (UAT) environment or a certification (CERT) environment. Theseenvironments are production like environments and ensure that the defectis first reproducible, and then apply the fix or solution, and then testif the fix is working along with the regression around the fix.

The continuous integration system (CIS) 205 facilitates the integrationwith the various systems to ensure that the right environment isprovided to the right test cases so that the result could beappropriate. In some embodiments, the CIS 205 also works on the resultbased approach. Thus, in case of failure the engine alerts back thedevelopment team to inform that a code fix is not working, and if foundworking ensures that the loop is closed and the production ready code isbuilt and ready for deployment. The CIS 205 gives an option to program aset of pilot stores, that is, if the data is programmed the solution ispiloted to the specific store(s).

As will be appreciated, the automatic execution of specific test casesmay be considered as an ideal situation. However, in some embodiments,the execution of test cases has to be done manually because of certainconstraints in the environment or certain limitations in automating theexecution of test cases. The execution module 204 may notify a user ifsuch instances occur. The execution module 204 may further provide areport that shows up the result of execution in a graphical formatindicating repeated occurrences of such instances. For example, in someembodiments, the published report may include a graph that indicates toa business user how many test cases are run manually and how many arerun automatically and how many are partially run on a quarterly basis.The report therefore provides an insight to the organization and enablesbusiness users to infer how effective is the automation, is theautomation happening at the right place or not, are there environmentfailures which hamper the automation, does the area which gets highestticket need to be automated, what test cases may or should be automated,how the errors are being treated, and so forth. The frequency of suchreports may be decided by the team.

Further, in some embodiments, the report may include a return oninvestment (ROI) analysis graph. It helps in providing to the business aview by ticket of what has happened in each and every incident so thatbusiness may ensure that the right incident gets the right attention.Based on the ROI analysis the funding every quarter may be adjusted bythe business that could result in better ROI for automation.Additionally, the report may include a comparative analysis of theresult which would enable the business user to determine for whichsolution there was more manual intervention needed and how furtherlearning may be done to improve such manual intervention.

It should be noted that the incident management engine 200 may beimplemented in programmable hardware devices such as programmable gatearrays, programmable array logic, programmable logic devices, and soforth. Alternatively, the incident management engine 200 may beimplemented in software for execution by various types of processors. Anidentified engine of executable code may, for instance, comprise one ormore physical or logical blocks of computer instructions which may, forinstance, be organized as an object, procedure, function, module, orother construct. Nevertheless, the executables of an identified engineneed not be physically located together, but may comprise disparateinstructions stored in different locations which, when joined logicallytogether, comprise the engine and achieve the stated purpose of theengine. Indeed, an engine of executable code could be a singleinstruction, or many instructions, and may even be distributed overseveral different code segments, among different applications, andacross several memory devices.

As will be appreciated by one skilled in the art, a variety of processesmay be employed for optimizing testing of software production incidents.For example, the exemplary system 100 and the associated incidentmanagement engine 200 may optimize testing of software productionincidents by the processes discussed herein. In particular, as will beappreciated by those of ordinary skill in the art, control logic and/orautomated routines for performing the techniques and steps describedherein may be implemented by the system 100 and the associated incidentmanagement engine 200, either by hardware, software, or combinations ofhardware and software. For example, suitable code may be accessed andexecuted by the one or more processors on the system 100 to perform someor all of the techniques described herein. Similarly, applicationspecific integrated circuits (ASICs) configured to perform some or allof the processes described herein may be included in the one or moreprocessors on the system 100.

For example, referring now to FIG. 3, exemplary control logic 300 foroptimizing testing of software production incidents via a system, suchas system 100, is depicted via a flowchart in accordance with someembodiments of the present disclosure. As illustrated in the flowchart,the control logic 300 includes the step of categorizing an incidentticket received from one or more sources based on one or morepre-defined parameters at step 301. In response to categorization, thecontrol logic 300 further includes the steps of analyzing the incidentticket using a machine learning algorithm to identify one or morekeywords in the incident ticket at step 302, and identifying a locationof the incident ticket based on the one or more keywords, a testworkspace corresponding to the incident ticket based on the location,and a plurality of specific test cases corresponding to the incidentticket based on the test workspace at step 303. The identification leadsto a first scenario and a second scenario. In the first scenario, thecontrol logic 300 further includes the step of initiating a learningprocess based on intelligence gathered from a manual processing of theincident ticket at step 304. In the second scenario, the control logic300 further includes the steps of identifying a test environment for theplurality of specific test cases at step 305, and executing theplurality of specific test cases in the test environment at step 306.

As noted above, the incident ticket corresponds to an obstruction (i.e.,a defect, or a failure) in a software production. Further, the one ormore predefined parameters comprise at least one of a person, a system,and a location related to the incident ticket. In some embodiments, thefirst scenario corresponds to a negative identification of at least oneof the location, the test workspace, and the plurality of specific testcases, while the second scenario corresponds to a positiveidentification of the location, the test workspace, and the plurality ofspecific test cases. Further, in some embodiments, the incident ticketresulting in the first scenario comprises a new incident ticketunrelated to a plurality of past incident tickets and not having atleast one of a corresponding location, a corresponding test workspace,and a corresponding specific test case. The manual processing of suchnew incident ticket comprises generating at least one of a solution, alocation, a test workspace, a test case, and a test environment.

In some embodiments, the control logic 300 further includes the step oflogging the incident ticket in an incident repository. Additionally, insome embodiments, the control logic 300 includes the step of routing theincident ticket to the identified location based on at least one of apriority and a severity rating of the one or more keywords and theidentified location. Further, in some embodiments, the control logic 300includes the step of verifying the plurality of specific test cases forsuitability to testing of the incident ticket. In some embodiments, thecontrol logic 300 further includes the step of updating at least one ofa location repository, a test workspace repository, a test caserepository, a keyword-location mapping, a location-workspace mapping,and a search keyword mapping based on the learning process. Moreover, insome embodiments, the control logic 300 includes the step of generatinga report indicating at least one of a result of the execution, a causeleading to failure of the execution, an effectiveness of implementationof the optimized testing, an area with high number of incident ticketsthat require implementation of the optimized testing, and a return oninvestment analysis related to implementation of the optimized testing.

In some embodiments, identifying the location of the incident ticket atstep 303 comprises referring to a keyword-location mapping. Similarly,in some embodiments, identifying the test workspace corresponding to theincident ticket at step 303 comprises referring to a location-workspacemapping. Additionally, in some embodiments, identifying the plurality ofspecific test cases at step 303 comprises identifying a test caselocation from a plurality of test workspaces by referring to alocation-workspace mapping and a search keyword mapping, and identifyingthe plurality of specific test cases from the test case location. Insome embodiments, identifying the plurality of specific test cases atstep 303 comprises analyzing a plurality of test cases in the testworkspace based on the one or more keywords using the machine learningalgorithm. Further, in some embodiments, identifying the testenvironment at step 305 comprises preparing the test environmentcorresponding to an environment of the software production and based onthe plurality of specific test cases.

Referring now to FIG. 4, exemplary control logic 400 for optimizingtesting of software production incidents is depicted in greater detailvia a flowchart in accordance with some embodiments of the presentdisclosure. As illustrated in the flowchart, the control logic 400includes the step of logging incident tickets and categorizing incidenttickets using artificial intelligence technique at step 401. The controllogic 400 further includes the step of checking with triage team anddetermining if data with respect to the incident ticket is available atstep 402. The data may include location details corresponding to theincident ticket and test workspace details corresponding to the locationdetails. The step 402 involves referring to the keyword-location mappingand the location-workspace mapping. If the data is available at step402, the control logic 400 further includes the step of determining ifthe content of the available data is sufficient for performing a searchat step 403. If the content is not sufficient at step 403, the controllogic 400 further includes the steps of requesting triage team toprovide more details or content with respect to the data at step 404 andchecking if more details are available with the triage team at step 405.If the content is sufficient at step 403 or if more details areavailable at step 405, the control logic 400 includes the steps ofsearching for required or specific test cases at step 406 anddetermining their applicability or suitability for testing of theincident tickets at step 407. If the test cases are applicable at step407, the control logic 400 includes the step of determining ifautomation is available for the test cases at step 408. If automation isavailable at step 408, the control logic 400 includes the step ofexecuting automation suite and validating results at step 409. However,if automation is not available at step 408, the control logic 400includes the step of executing manual suite and validating results atstep 410. Moreover, if the data is not available at step 402, or if moredetails are not available at step 405, or if the test cases are notapplicable at step 407, the control logic 400 includes the step ofstarting self-learning engine to learn from manual input process at step411.

As will be also appreciated, the above described techniques may take theform of computer or controller implemented processes and apparatuses forpracticing those processes. The disclosure can also be embodied in theform of computer program code containing instructions embodied intangible media, such as floppy diskettes, CD-ROMs, hard drives, or anyother computer-readable storage medium, wherein, when the computerprogram code is loaded into and executed by a computer or controller,the computer becomes an apparatus for practicing the invention. Thedisclosure may also be embodied in the form of computer program code orsignal, for example, whether stored in a storage medium, loaded intoand/or executed by a computer or controller, or transmitted over sometransmission medium, such as over electrical wiring or cabling, throughfiber optics, or via electromagnetic radiation, wherein, when thecomputer program code is loaded into and executed by a computer, thecomputer becomes an apparatus for practicing the invention. Whenimplemented on a general-purpose microprocessor, the computer programcode segments configure the microprocessor to create specific logiccircuits.

The disclosed methods and systems may be implemented on a conventionalor a general-purpose computer system, such as a personal computer (PC)or server computer. Referring now to FIG. 5, a block diagram of anexemplary computer system 501 for implementing embodiments consistentwith the present disclosure is illustrated. Variations of computersystem 501 may be used for implementing system 100 and incidentmanagement engine 200 for optimizing testing of the software productionincidents. Computer system 501 may comprise a central processing unit(“CPU” or “processor”) 502. Processor 502 may comprise at least one dataprocessor for executing program components for executing user- orsystem-generated requests. A user may include a person, a person using adevice such as such as those included in this disclosure, or such adevice itself. The processor may include specialized processing unitssuch as integrated system (bus) controllers, memory management controlunits, floating point units, graphics processing units, digital signalprocessing units, etc. The processor may include a microprocessor, suchas AMD Athlon, Duron or Opteron, ARM's application, embedded or secureprocessors, IBM PowerPC, Intel's Core, Itanium, Xeon, Celeron or otherline of processors, etc. The processor 502 may be implemented usingmainframe, distributed processor, multi-core, parallel, grid, or otherarchitectures. Some embodiments may utilize embedded technologies likeapplication-specific integrated circuits (ASICs), digital signalprocessors (DSPs), Field Programmable Gate Arrays (FPGAs), etc.

Processor 502 may be disposed in communication with one or moreinput/output (I/O) devices via I/O interface 503. The I/O interface 503may employ communication protocols/methods such as, without limitation,audio, analog, digital, monoaural, RCA, stereo, IEEE-1394, serial bus,universal serial bus (USB), infrared, PS/2, BNC, coaxial, component,composite, digital visual interface (DVI), high-definition multimediainterface (HDMI), RF antennas, S-Video, VGA, IEEE 802.n/b/g/n/x,Bluetooth, cellular (e.g., code-division multiple access (CDMA),high-speed packet access (HSPA+), global system for mobilecommunications (GSM), long-term evolution (LTE), WiMax, or the like),etc.

Using the I/O interface 503, the computer system 501 may communicatewith one or more I/O devices. For example, the input device 504 may bean antenna, keyboard, mouse, joystick, (infrared) remote control,camera, card reader, fax machine, dongle, biometric reader, microphone,touch screen, touchpad, trackball, sensor (e.g., accelerometer, lightsensor, GPS, gyroscope, proximity sensor, or the like), stylus, scanner,storage device, transceiver, video device/source, visors, etc. Outputdevice 505 may be a printer, fax machine, video display (e.g., cathoderay tube (CRT), liquid crystal display (LCD), light-emitting diode(LED), plasma, or the like), audio speaker, etc. In some embodiments, atransceiver 506 may be disposed in connection with the processor 502.The transceiver may facilitate various types of wireless transmission orreception. For example, the transceiver may include an antennaoperatively connected to a transceiver chip (e.g., Texas InstrumentsWiLink WL1283, Broadcom BCM4750IUB8, Infineon Technologies X-Gold618-PMB9800, or the like), providing IEEE 802.11a/b/g/n, Bluetooth, FM,global positioning system (GPS), 2G/3G HSDPA/HSUPA communications, etc.

In some embodiments, the processor 502 may be disposed in communicationwith a communication network 508 via a network interface 507. Thenetwork interface 507 may communicate with the communication network508. The network interface may employ connection protocols including,without limitation, direct connect, Ethernet (e.g., twisted pair10/100/1000 Base T), transmission control protocol/internet protocol(TCP/IP), token ring, IEEE 802.11a/b/g/n/x, etc. The communicationnetwork 508 may include, without limitation, a direct interconnection,local area network (LAN), wide area network (WAN), wireless network(e.g., using Wireless Application Protocol), the Internet, etc. Usingthe network interface 507 and the communication network 508, thecomputer system 501 may communicate with devices 509, 510, and 511.These devices may include, without limitation, personal computer(s),server(s), fax machines, printers, scanners, various mobile devices suchas cellular telephones, smartphones (e.g., Apple iPhone, Blackberry,Android-based phones, etc.), tablet computers, eBook readers (AmazonKindle, Nook, etc.), laptop computers, notebooks, gaming consoles(Microsoft Xbox, Nintendo DS, Sony PlayStation, etc.), or the like. Insome embodiments, the computer system 501 may itself embody one or moreof these devices.

In some embodiments, the processor 502 may be disposed in communicationwith one or more memory devices (e.g., RAM 413, ROM 414, etc.) via astorage interface 512. The storage interface may connect to memorydevices including, without limitation, memory drives, removable discdrives, etc., employing connection protocols such as serial advancedtechnology attachment (SATA), integrated drive electronics (IDE),IEEE-1394, universal serial bus (USB), fiber channel, small computersystems interface (SCSI), etc. The memory drives may further include adrum, magnetic disc drive, magneto-optical drive, optical drive,redundant array of independent discs (RAID), solid-state memory devices,solid-state drives, etc.

The memory devices may store a collection of program or databasecomponents, including, without limitation, an operating system 516, userinterface application 517, web browser 518, mail server 519, mail client520, user/application data 521 (e.g., any data variables or data recordsdiscussed in this disclosure), etc. The operating system 516 mayfacilitate resource management and operation of the computer system 501.Examples of operating systems include, without limitation, AppleMacintosh OS X, Unix, Unix-like system distributions (e.g., BerkeleySoftware Distribution (BSD), FreeBSD, NetBSD, OpenBSD, etc.), Linuxdistributions (e.g., Red Hat, Ubuntu, Kubuntu, etc.), IBM OS/2,Microsoft Windows (XP, Vista/7/8, etc.), Apple iOS, Google Android,Blackberry OS, or the like. User interface 517 may facilitate display,execution, interaction, manipulation, or operation of program componentsthrough textual or graphical facilities. For example, user interfacesmay provide computer interaction interface elements on a display systemoperatively connected to the computer system 501, such as cursors,icons, check boxes, menus, scrollers, windows, widgets, etc. Graphicaluser interfaces (GUIs) may be employed, including, without limitation,Apple Macintosh operating systems' Aqua, IBM OS/2, Microsoft Windows(e.g., Aero, Metro, etc.), Unix X-Windows, web interface libraries(e.g., ActiveX, Java, Javascript, AJAX, HTML, Adobe Flash, etc.), or thelike.

In some embodiments, the computer system 501 may implement a web browser518 stored program component. The web browser may be a hypertext viewingapplication, such as Microsoft Internet Explorer, Google Chrome, MozillaFirefox, Apple Safari, etc. Secure web browsing may be provided usingHTTPS (secure hypertext transport protocol), secure sockets layer (SSL),Transport Layer Security (TLS), etc. Web browsers may utilize facilitiessuch as AJAX, DHTML, Adobe Flash, JavaScript, Java, applicationprogramming interfaces (APIs), etc. In some embodiments, the computersystem 501 may implement a mail server 519 stored program component. Themail server may be an Internet mail server such as Microsoft Exchange,or the like. The mail server may utilize facilities such as ASP,ActiveX, ANSI C++/C#, Microsoft .NET, CGI scripts, Java, JavaScript,PERL, PHP, Python, WebObjects, etc. The mail server may utilizecommunication protocols such as internet message access protocol (IMAP),messaging application programming interface (MAPI), Microsoft Exchange,post office protocol (POP), simple mail transfer protocol (SMTP), or thelike. In some embodiments, the computer system 501 may implement a mailclient 520 stored program component. The mail client may be a mailviewing application, such as Apple Mail, Microsoft Entourage, MicrosoftOutlook, Mozilla Thunderbird, etc.

In some embodiments, computer system 501 may store user/application data521, such as the data, variables, records, etc. (e.g., incident tickets,locations/areas, priority and severity ratings, keywords, testworkspaces, location-workspace mapping, search keyword mapping, testcases, and so forth) as described in this disclosure. Such databases maybe implemented as fault-tolerant, relational, scalable, secure databasessuch as Oracle or Sybase. Alternatively, such databases may beimplemented using standardized data structures, such as an array, hash,linked list, struct, structured text file (e.g., XML), table, or asobject-oriented databases (e.g., using ObjectStore, Poet, Zope, etc.).Such databases may be consolidated or distributed, sometimes among thevarious computer systems discussed above in this disclosure. It is to beunderstood that the structure and operation of the any computer ordatabase component may be combined, consolidated, or distributed in anyworking combination.

As will be appreciated by those skilled in the art, the techniquesdescribed in the various embodiments discussed above results inefficient, robust, and cost-effective management and testing of softwareproduction incidents subsequent to release of a software product. Thetechniques described in the embodiments discussed above provide anautomated process of testing a defect that is found in production,thereby ensuring a consistent and predictive delivery of softwareproduct quality. The self-learning mechanism of the process ensures thatthe process keeps improving its efficiency after putting to use. Thefeedback mechanism of the process provides feedback to user with respectto benefits and the areas of improvement. The feedback mechanism furtherreceives feedback from users for continuous improvement of the process.Additionally, the techniques described in the embodiments discussedabove analyzes the production defects and learns from the pattern,correlates the defects with the existing test cases, and ensures asmooth build, run, and reinstallation into production. Further, thetechniques described in the embodiments discussed above is easy to buildand use and can be integrated with any system.

The specification has described system and method for optimizing testingof software production incidents. The illustrated steps are set out toexplain the exemplary embodiments shown, and it should be anticipatedthat ongoing technological development will change the manner in whichparticular functions are performed. These examples are presented hereinfor purposes of illustration, and not limitation. Further, theboundaries of the functional building blocks have been arbitrarilydefined herein for the convenience of the description. Alternativeboundaries can be defined so long as the specified functions andrelationships thereof are appropriately performed. Alternatives(including equivalents, extensions, variations, deviations, etc., ofthose described herein) will be apparent to persons skilled in therelevant art(s) based on the teachings contained herein. Suchalternatives fall within the scope and spirit of the disclosedembodiments.

Furthermore, one or more computer-readable storage media may be utilizedin implementing embodiments consistent with the present disclosure. Acomputer-readable storage medium refers to any type of physical memoryon which information or data readable by a processor may be stored.Thus, a computer-readable storage medium may store instructions forexecution by one or more processors, including instructions for causingthe processor(s) to perform steps or stages consistent with theembodiments described herein. The term “computer-readable medium” shouldbe understood to include tangible items and exclude carrier waves andtransient signals, i.e., be non-transitory. Examples include randomaccess memory (RAM), read-only memory (ROM), volatile memory,nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, andany other known physical storage media.

It is intended that the disclosure and examples be considered asexemplary only, with a true scope and spirit of disclosed embodimentsbeing indicated by the following claims.

1. A method for optimizing testing of software production incidents, themethod comprising: categorizing, via a processor, an incident ticketreceived from one or more sources based on one or more pre-definedparameters, the incident ticket corresponding to an obstruction in asoftware production; in response to categorization, analyzing, via theprocessor, the incident ticket using a machine learning algorithm toidentify one or more keywords in the incident ticket; identifying, viathe processor, a location of the incident ticket based on the one ormore keywords, a test workspace corresponding to the incident ticketbased on the location, and a plurality of specific test casescorresponding to the incident ticket based on the test workspace, theidentification leading to a first scenario and a second scenario; in thefirst scenario, initiating, via the processor, a learning process basedon intelligence gathered from a manual processing of the incidentticket; and in the second scenario, identifying, via the processor, atest environment for the plurality of specific test cases; andexecuting, via the processor, the plurality of specific test cases inthe test environment, wherein the first scenario corresponds to anegative identification of at least one of the location, the testworkspace, and the plurality of specific test cases, wherein the secondscenario corresponds to a positive identification of the location, thetest workspace, and the plurality of specific test cases, wherein theincident ticket resulting in the first scenario comprises a new incidentticket unrelated to a plurality of cast incident tickets and not havingat least one of a corresponding location, a corresponding testworkspace, and a corresponding specific test case, and wherein themanual processing of the new incident ticket comprises generating atleast one of a solution, a location, a test workspace, a test case, anda test environment.
 2. The method of claim 1, further comprising loggingthe incident ticket in an incident repository.
 3. The method of claim 1,wherein the one or more predefined parameters comprise at least one of aperson, a system, and a location related to the incident ticket.
 4. Themethod of claim 1, wherein identifying the location of the incidentticket comprises referring to a keyword-location mapping.
 5. The methodof claim 1, further comprising routing the incident ticket to theidentified location based on at least one of a priority and a severityrating of the one or more keywords and the identified location.
 6. Themethod of claim 1, wherein identifying the test workspace correspondingto the incident ticket comprises referring to a location-workspacemapping.
 7. The method of claim 1, wherein identifying the plurality ofspecific test cases comprises: identifying a test case location from aplurality of test workspaces by referring to a location-workspacemapping and a search keyword mapping; and identifying the plurality ofspecific test cases from the test case location.
 8. The method of claim1, wherein identifying the plurality of specific test cases comprisesanalyzing a plurality of test cases in the test workspace based on theone or more keywords using the machine learning algorithm.
 9. The methodof claim 1, further comprising verifying the plurality of specific testcases for suitability to testing of the incident ticket.
 10. (canceled)11. The method of claim 1, wherein identifying the test environmentcomprises preparing the test environment corresponding to an environmentof the software production and based on the plurality of specific testcases.
 12. (canceled)
 13. The method of claim 1, further comprisingupdating at least one of a location repository, a test workspacerepository, a test case repository, a keyword-location mapping, alocation-workspace mapping, and a search keyword mapping based on thelearning process.
 14. The method of claim 1, further comprisinggenerating a report indicating at least one of a result of theexecution, a cause leading to failure of the execution, an effectivenessof implementation of the optimized testing, an area with high number ofincident tickets that require implementation of the optimized testing,and a return on investment analysis related to implementation of theoptimized testing.
 15. A system for optimizing testing of softwareproduction incidents, the system comprising: at least one processor; anda computer-readable medium storing instructions that, when executed bythe at least one processor, cause the at least one processor to performoperations comprising: categorizing an incident ticket received from oneor more sources based on one or more pre-defined parameters, theincident ticket corresponding to an obstruction in a softwareproduction; in response to categorization, analyzing the incident ticketusing a machine learning algorithm to identify one or more keywords inthe incident ticket; identifying a location of the incident ticket basedon the one or more keywords, a test workspace corresponding to theincident ticket based on the location, and a plurality of specific testcases corresponding to the incident ticket based on the test workspace,the identification leading to a first scenario and a second scenario; inthe first scenario, initiating a learning process based on intelligencegathered from a manual processing of the incident ticket; and in thesecond scenario, identifying a test environment for the plurality ofspecific test cases, and executing the plurality of specific test casesin the test environment, wherein the first scenario corresponds to anegative identification of at least one of the location, the testworkspace, and the plurality of specific test cases, wherein the secondscenario corresponds to a positive identification of the location, thetest workspace, and the plurality of specific test cases, wherein theincident ticket resulting in the first scenario comprises a new incidentticket unrelated to a plurality of past incident tickets and not havingat least one of a corresponding location, a corresponding testworkspace, and a corresponding specific test case, and wherein themanual processing of the new incident ticket comprises generating atleast one of a solution, a location, a test workspace, a test case, anda test environment.
 16. The system of claim 15, wherein identifying thelocation of the incident ticket comprises referring to akeyword-location mapping, and wherein identifying the test workspacecorresponding to the incident ticket comprises referring to alocation-workspace mapping, and wherein identifying the plurality ofspecific test cases comprises identifying a test case location from aplurality of test workspaces by referring to a location-workspacemapping and a search keyword mapping and identifying the plurality ofspecific test cases from the test case location.
 17. (canceled)
 18. Thesystem of claim 15, wherein the manual processing of the incident ticketcomprises generating at least one of a solution, a location, a testworkspace, a test case, and a test environment, and wherein theoperations further comprise updating at least one of a locationrepository, a test workspace repository, a test case repository, akeyword-location mapping, a location-workspace mapping, and a searchkeyword mapping based on the learning process.
 19. A non-transitorycomputer-readable medium storing computer-executable instructions for:categorizing an incident ticket received from one or more sources basedon one or more pre-defined parameters, the incident ticket correspondingto an obstruction in a software production; in response tocategorization, analyzing the incident ticket using a machine learningalgorithm to identify one or more keywords in the incident ticket;identifying a location of the incident ticket based on the one or morekeywords, a test workspace corresponding to the incident ticket based onthe location, and a plurality of specific test cases corresponding tothe incident ticket based on the test workspace, the identificationleading to a first scenario and a second scenario; in the firstscenario, initiating a learning process based on Intelligence gatheredfrom a manual processing of the incident ticket; and in the secondscenario, identifying a test environment for the plurality of specifictest cases, and executing the plurality of specific test cases in thetest environment, wherein the first scenario corresponds to a negativeidentification of at least one of the location, the test workspace, andthe plurality of specific test cases, wherein the second scenariocorresponds to a positive identification of the location, the testworkspace, and the plurality of specific test cases, wherein theincident ticket resulting in the first scenario comprises a new incidentticket unrelated to a plurality of past incident tickets and not havingat least one of a corresponding location, a corresponding testworkspace, and a corresponding specific test case, and wherein themanual processing of the new incident ticket comprises generating atleast one of a solution, a location, a test workspace, a test case, anda test environment.